Skip to content

feat(sdk): add extension auth for middleware E2E - #2784

Open
pimlock wants to merge 5 commits into
mainfrom
1733-supervisor-middleware-e2e-sdk/pimlock
Open

feat(sdk): add extension auth for middleware E2E#2784
pimlock wants to merge 5 commits into
mainfrom
1733-supervisor-middleware-e2e-sdk/pimlock

Conversation

@pimlock

@pimlock pimlock commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Establish the first Rust SDK surface for building OpenShell extension services and add authenticated supervisor middleware end-to-end coverage.

This PR intentionally starts with a narrow foundation: the opt-in openshell-sdk extension feature provides service-side caller verification, and one Docker E2E proves the complete gateway → sandbox supervisor → authenticated middleware → upstream HTTP path. The middleware replaces sensitive content and adds a header before the request reaches the upstream.

Related Issue

Related to #1733.

Follow-up coverage and SDK adoption are tracked in #2794.

SDK Direction

The extension module is intended to become the Rust SDK for implementing extension services. As it grows, the SDK should own common lifecycle concerns such as service interfaces, authentication before handler dispatch, key caching and refresh after trusted bootstrap, protocol lifecycle validation, consistent errors, and observability. Extension authors should primarily implement manifests, configuration validation, and request or message business logic.

Rust is the initial proving ground, not a Rust-only extension model. OpenShell's other language SDKs should provide equivalent extension support within their existing packages rather than introducing a separate family of extension packages. The cross-language API shape still needs dedicated design work so each SDK remains idiomatic while sharing the same lifecycle, trust, and interoperability guarantees.

The deployment must continue to provide cold-start trust configuration, including the trusted gateway identity, issuer, audience, and CA or initial verification keys. Extension-specific authorization also remains application-owned. In this initial PR, key refresh remains deployment-owned and middleware protobuf interfaces still come from the existing core surface.

Alternative Considered

We considered publishing a dedicated extension SDK, separate from the existing API SDK in each language. Keeping extension support in the regular SDKs gives OpenShell fewer packages, release paths, and dependency surfaces to maintain. It also reflects the expected usage model: extensions will likely call OpenShell APIs in addition to implementing service interfaces.

A unified SDK can eventually provide an extension-oriented API client wrapper with the appropriate authentication configured for extension-to-OpenShell calls. This avoids requiring extension authors to combine and configure two SDK families themselves. Cargo features and language-appropriate modules can preserve an opt-in boundary without introducing separate packages.

Changes

  • Extension SDK: Add the opt-in openshell-sdk extension feature with self-contained caller authentication from trusted Ed25519 keys or JWKS, strict token and identity validation, and typed authenticated caller information without a direct openshell-extension-core dependency.
  • Authenticated middleware E2E: Add a TLS fixture built against the new SDK feature and one Docker happy path covering gateway and supervisor authentication, request transformation, local upstream verification, ephemeral trust setup, a dedicated mise task, and a GitHub Actions E2E lane.

Follow-up

Issue #2794 tracks the next layer of work:

  • migrate examples/supervisor-middleware-content-guard to the SDK and use it as the behavioral E2E fixture
  • add HTTP and WebSocket redact/deny coverage, invalid configuration, failure modes, chaining, and payload-limit scenarios
  • add direct negative verifier tests for issuer, audience, signing key, kid, algorithm, time bounds, claim shape, and JWKS validation
  • require authentication even on unsupported fixture RPC stubs

Testing

  • mise run pre-commit passes
  • SDK extension authentication unit tests pass
  • Standalone middleware fixture builds with its locked dependency graph
  • mise run e2e:supervisor-middleware passes against a Docker-backed gateway

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

@copy-pr-bot

copy-pr-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown

@pimlock pimlock self-assigned this Aug 18, 2026
@pimlock pimlock added the test:e2e Requires end-to-end coverage label Aug 18, 2026
@pimlock

pimlock commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test 3128169

@github-actions

Copy link
Copy Markdown

Label test:e2e applied for 3128169. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@pimlock
pimlock marked this pull request as ready for review August 18, 2026 23:22
@pimlock

This comment was marked as outdated.

@pimlock pimlock added the gator:blocked Gator is blocked by process or repository gates label Aug 20, 2026
@pimlock
pimlock force-pushed the 1733-supervisor-middleware-e2e-sdk/pimlock branch from eb6130a to 02ab41b Compare August 20, 2026 23:26
@pimlock pimlock added gator:watch-pipeline Gator is monitoring PR CI/CD status and removed gator:blocked Gator is blocked by process or repository gates labels Aug 20, 2026
@pimlock

This comment was marked as outdated.

@pimlock pimlock added gator:blocked Gator is blocked by process or repository gates and removed gator:watch-pipeline Gator is monitoring PR CI/CD status labels Aug 21, 2026
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
@pimlock
pimlock force-pushed the 1733-supervisor-middleware-e2e-sdk/pimlock branch from 02ab41b to 79f4857 Compare August 21, 2026 16:29
@pimlock

This comment was marked as outdated.

@pimlock pimlock added gator:watch-pipeline Gator is monitoring PR CI/CD status gator:blocked Gator is blocked by process or repository gates and removed gator:blocked Gator is blocked by process or repository gates gator:watch-pipeline Gator is monitoring PR CI/CD status labels Aug 21, 2026
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
@pimlock

pimlock commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

gator-agent

PR Review Status

Validation: Project-valid maintainer-authored work implementing the authenticated supervisor-middleware SDK direction tracked by #1733, with follow-up scope tracked in #2794.
Head SHA: 73d2246d04cb8ec7d4477905abb99f7feb27b20b
Base SHA: de4c1fecf564cd16854447ec9c08659e7b64061b
Merge base SHA: de4c1fecf564cd16854447ec9c08659e7b64061b
Patch ID: 16df71b113135671297e287b5b39b41d8db7c6ad
Gator payload: 5
Review mode: follow_up
Previous reviewed SHA: 79f4857fd7df23cde5d602ce9a850a715801bfbb
Review budget exhausted: no
Maintainer decision required: no

Blocking findings:

  • No blocking findings remain.

Carried findings:

  • None.

Docs: The SDK README and relevant Fern supervisor-middleware documentation remain updated; this lockfile-only delta does not require additional docs or navigation changes.

Checks: DCO, Helm Lint, and GPU E2E are green. The current-head Branch Checks and required OpenShell / E2E workflow are dispatched and pending. Gator did not rerun any workflow, post /ok to test, change test labels, approve, merge, or push.

Next state: gator:watch-pipeline

@pimlock pimlock added gator:watch-pipeline Gator is monitoring PR CI/CD status gator:approval-needed Gator completed review; maintainer approval needed and removed gator:blocked Gator is blocked by process or repository gates gator:watch-pipeline Gator is monitoring PR CI/CD status labels Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gator:approval-needed Gator completed review; maintainer approval needed test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant